Search for: All records

Preferences within a group of people are not uniform but follow a distribution. While existing alignment methods like Direct Preference Optimization (DPO) attempt to steer models to reflect human preferences, they struggle to capture the distributional pluralistic preferences within a group. These methods often skew toward dominant preferences, overlooking the diversity of opinions, especially when conflicting preferences arise. To address this issue, we propose Group Distributional Preference Optimization (GDPO), a novel framework that aligns language models with the distribution of preferences within a group by incorporating the concept of beliefs that shape individual preferences. GDPO calibrates a language model using statistical estimation of the group's belief distribution and aligns the model with belief-conditioned preferences, offering a more inclusive alignment framework than traditional methods. In experiments using both synthetic controllable opinion generation and real-world movie review datasets, we show that DPO fails to align with the targeted belief distributions, while GDPO consistently reduces this alignment gap during training. Additionally, our evaluation metrics demonstrate that GDPO outperforms existing approaches in aligning with group distributional preferences, marking a significant advance in pluralistic alignment. 

Abstract If Type Ia supernovae (SNe Ia) result from a white dwarf being ignited by Roche-lobe overflow from a nondegenerate companion, then as the SN explosion runs into the companion star its ejecta will be shocked, causing an early blue excess in the lightcurve. A handful of these excesses have been found in single-object studies, but inferences about the population of SNe Ia as a whole have been limited because of the rarity of multiwavelength follow-up within days of explosion. Here we present a 3 yr investigation yielding a nearly unbiased sample of nine nearby (z < 0.01) SNe Ia with exemplary early data. The data are multiwavelength, coveringUBVgriand Neil Gehrels Swift Observatory UV bandpasses, and also early, with an average first epoch 16.0 days before maximum light. Of the nine objects, three show early blue excesses. We do not find enough statistical evidence to reject the null hypothesis that SNe Ia predominantly arise from Roche-lobe-overflowing single-degenerate systems (p= 0.94). When looking at the objects’ colors, we find the objects are almost uniformly near-UV–blue, in contrast to earlier literature samples which found that only a third of SNe Ia are near-UV–blue, and we find a seemingly continuous range ofB − Vcolors in the days after explosion, again in contrast with earlier claims in the literature. This study highlights the importance of early, multiwavelength, high-cadence data in determining the progenitor systems of SNe Ia and in revealing their diverse early behavior. 

Finetuned large language models (LLMs) have shown remarkable performance in financial tasks, such as sentiment analysis and information retrieval. Due to privacy concerns, finetuning and deploying financial LLMs (FinLLMs) locally are crucial for institutions and individuals. In this paper, we employ quantized low-rank adaptation (QLoRA) to finetune FinLLMs, which leverage low-rank structure and quantization technique to significantly reduce computational requirements while maintaining model performance. We also employ data and pipeline parallelism to enable local finetuning on commodity GPUs. Experiments on financial datasets validate the efficacy of our approach in yielding notable improvements over the base models. 

Large Vision-Language Models (LVLMs) have made substantial progress by integrating pre-trained large language models (LLMs) and vision models through instruction tuning. Despite these advancements, LVLMs often exhibit the hallucination phenomenon, where generated text responses appear linguistically plausible but contradict the input image, indicating a misalignment between image and text pairs. This misalignment arises because the model tends to prioritize textual information over visual input, even when both the language model and visual representations are of high quality. Existing methods leverage additional models or human annotations to curate preference data and enhance modality alignment through preference optimization. These approaches are resource-intensive and may not effectively reflect the target LVLM's preferences, making the curated preferences easily distinguishable. Our work addresses these challenges by proposing the Calibrated Self-Rewarding (CSR) approach, which enables the model to self-improve by iteratively generating candidate responses, evaluating the reward for each response, and curating preference data for fine-tuning. In the reward modeling, we employ a step-wise strategy and incorporate visual constraints into the self-rewarding process to place greater emphasis on visual input. Empirical results demonstrate that CSR significantly enhances performance and reduces hallucinations across twelve benchmarks and tasks, achieving substantial improvements over existing methods by 7.62%. Our empirical results are further supported by rigorous theoretical analysis, under mild assumptions, verifying the effectiveness of introducing visual constraints into the self-rewarding paradigm. Additionally, CSR shows compatibility with different vision-language models and the ability to incrementally improve performance through iterative fine-tuning. 

How do practitioners who develop consumer AI products scope, motivate, and conduct privacy work? Respecting pri- vacy is a key principle for developing ethical, human-centered AI systems, but we cannot hope to better support practitioners without answers to that question. We interviewed 35 industry AI practitioners to bridge that gap. We found that practitioners viewed privacy as actions taken against pre-defined intrusions that can be exacerbated by the capabilities and requirements of AI, but few were aware of AI-specific privacy intrusions documented in prior literature. We found that their privacy work was rigidly defined and situated, guided by compliance with privacy regulations and policies, and generally demoti- vated beyond meeting minimum requirements. Finally, we found that the methods, tools, and resources they used in their privacy work generally did not help address the unique pri- vacy risks introduced or exacerbated by their use of AI in their products. Collectively, these findings reveal the need and opportunity to create tools, resources, and support structures to improve practitioners’ awareness of AI-specific privacy risks, motivations to do AI privacy work, and ability to ad- dress privacy harms introduced or exacerbated by their use of AI in consumer products. 

Predicting the behavior of nanomaterials under various conditions presents a significant challenge due to their complex microstructures. While high-fidelity modeling techniques, such as molecular dynamics (MD) simulations, are effective, they are also computationally demanding. Machine learning (ML) models have opened new avenues for the rapid exploration of design spaces. In this work, we developed a deep learning framework based on a conditional generative adversarial network (cGAN) to predict the evolution of grain boundary (GB) networks in nanocrystalline materials under mechanical loads, incorporating both morphological and atomic details. We conducted MD simulations on nanocrystalline tungsten and used the resulting ground-truth data to train our cGAN model. We assessed the performance of our cGAN model by comparing it to a Convolutional Autoencoder (ConvAE) model and examining the impact of changes in geometric morphology and loading conditions on the model's performance. Our cGAN model demonstrated high accuracy in predicting GB network evolution under a variety of conditions. This developed framework shows potential for predicting various materials' behaviors across a wide range of nanomaterials.